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PREFACE 


The  work  described  in  this  report  was  authorized  under  Project  No. 

206023. 84BPO.  This  work  was  started  in  August  2001  and  completed  in  July  2003. 

The  use  of  either  trade  or  manufacturers'  names  in  this  report  does  not  constitute 
an  official  endorsement  of  any  commercial  products.  This  report  may  not  be  cited  for  purposes 
of  advertisement. 

The  report  has  been  approved  for  public  release.  Registered  users  should  request 
additional  copies  from  the  Defense  Technical  Information  Center;  unregistered  users  should 
direct  such  requests  to  the  National  Technical  Information  Service. 
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DETECTING  CHEMICAL  AND  BIOLOGICAL  MUNITIONS 
USING  INFRARED  IMAGING  SYSTEMS,  ADVANCED  IMAGE  PROCESSING  AND 
FEATURE  EXTRACTION  METHODS,  AND  LINGUISTIC-FUZZY  CLASSIFIERS 


1.  INTRODUCTION 

In  September  2001,  the  U.S.  Army  Edgewood  Chemical  Biological  Center 
(ECBC)  conducted  a  field  test  at  Dugway  Proving  Ground  (DPG),  Dugway,  UT.  The  test  was 
part  of  the  Disparate  Sensor  Integration  (DSI)  program.  The  purpose  of  the  program  is  to 
determine  if  chemical/biological  (CB)  munitions  could  be  detected,  based  on  their  different 
signatures  during  detonation.  During  the  test,  260  155mm  artillery  rounds  were  fired.  The 
classes  of  160  of  these  munitions  were  known  to  the  analyst  and  were  used  to  help  identify 
critical  features  and  develop  algorithms  to  detect  CB  munitions.  The  last  100  rounds  were  blind 
(to  the  analyst)  and  were  used  to  test  the  robustness  of  the  discriminating  algorithms.  The  rounds 
were  divided  equally  between  conventional  (HE)  and  simulated  CB  rounds.  The  conventional 
rounds  contained  15  lb  of  high  explosive  (HE),  while  the  CB  rounds  contained  about  1.5  lb  of 
HE  and  were  filled  with  a  50-50%  mixture  of  water  and  poly  ethylene  glycol.  Half  of  the 
munitions  were  fused  to  detonate  upon  ground  impact.  The  other  half  were  fused  for  airburst. 
The  rounds  were  fired  in  random  order.  Detailed  descriptions  of  the  test  and  the  various  sensors 
used  are  given  in  ECBC-TR-251.1 

In  support  of  the  DSI  Program,  several  camera  systems  were  fielded  to  determine 
the  feasibility  of  using  information  from  these  systems  to  discriminate  between  chemical  and 
conventional  munitions.  The  cameras  fielded  included  two  visible  wavelength  cameras,  a  near 
infrared  (1R)  camera  (peak  response  at  approximately  1  pm),  a  mid  wavelength  IR  camera 
system  (3  pm  -  5  pm),  and  three  long  wavelength  IR  camera  systems  (7.5  pm  -  13  pm). 

One  of  these  imaging  systems  was  a  U.S.  Air  Force-deployed  system  currently 
used  for  surveillance  on  the  battlefield.  The  Long  Wavelength  IR  Imaging  System  uses  an  un¬ 
cooled  silicon  microbolimeter  (320  x  240  pixels)  detector  array  that  is  sensitive  over  the 
7.5  -  13  pm  spectral  range.  This  system  is  relatively  inexpensive  compared  to  other  imaging 
sensors  evaluated.  More  importantly,  it  is  a  fielded  system  that  is  currently  used  on  the 
battlefield.  Other  reports2'3  summarized  the  results  from  preliminary  work  performed  on  data 
generated  from  this  and  a  visible  spectral  range  imaging  system.  These  reports  describe  some 
basic  features  with  potential  use  in  discriminating  between  the  different  classes  of  munitions' 
detonation  events.  This  work  also  demonstrated  the  potential  for  achieving  this  discrimination 
within  approximately  1/3  of  a  second  (10  frames)  from  detonation  time. 

This  report  describes  some  of  the  more  recent  work  performed  on  the  DSI  Air 
Force  IR  imaging  system  data  set.  Specifically,  this  report  will  first  detail  the  “detector"  used  to 
identify  munitions'  detonation  events  within  detonation  video  sequences  and  will  provide 
representative  imagery  from  each  of  the  four  classes  of  munitions  detonations  (HE  point 
detonation,  CB  point  detonation,  HE  airburst,  and  CB  airburst).  This  information  will  be 
followed  by  descriptions  of  the  features  extracted  from  frames  of  interest  within  the  video 
sequence.  Conversion  of  the  features  extracted  from  individual  frames  to  a  final  feature  vector 
(used  as  input  to  a  classification  system)  will  then  be  described.  The  linguistic-fuzzy  classifiers4 
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used  to  analyze  this  feature  vector  and  to  estimate  munition's  detonation  event  class  confidence 
will  be  described  in  detail.  A  discussion  of  the  results  obtained  on  the  DSI  training  and  blind 
data  sets  using  the  previously  described  methodologies  will  follow.  The  report  will  conclude 
with  recommendations  and  discussions  on  how  the  confidence  values  generated  and  feature 
values  can  be  used  to  support  DSI  sensor  and  data  fusion  activities. 


2.  DETECTOR  DESCRIPTION  AND  REPRESENTATIVE  IMAGERY 

This  section  of  the  report  will  first  describe  the  detector  used  to  identify  "frames” 
of  interest  in  the  DSI  video  sequences.  This  will  be  followed  by  representative  imagery  from 
each  of  the  four  classes  of  events: 

(1)  CB  airburst 

(2)  HE  airburst 

(3)  CB  ground  detonation 

(4)  HE  ground  detonation 

The  DSI  imaging  sensor  data  were  archived  and  made  available  to  DSI  team 
members  as  AVI  files  (compressed  with  various  CODECs).  These  files  typically  consist  of  a 
video  sequence  with  some  background  video  followed  by  the  munition's  detonation  event 
followed  by  some  additional  background  video.  The  events  of  interest  last  between  3  and  100 
(or  more)  frames  within  these  sequences.  Consequently,  we  wanted  to  develop  a  detector  that 
could  be  used  to  isolate  and  identify  frames  of  interest  within  the  video  sequence.  This  detector 
is  described  in  detail  in  the  following  paragraphs.  It  will  become  apparent  in  following 
discussions  that  a  secondary  benefit  of  this  detector  is  that  in  the  image,  it  also  identifies  the 
specific  pixel  locations  associated  with  the  munition's  detonation  event. 

Figure  1  provides  a  representative  frame  extracted  from  the  3rd  frame  after 
detonation  from  an  HE  airburst.  This  frame  is  converted  to  a  digital  grayscale  image  whose 
intensity  varies  from  0  (dark  or  "hot”)  to  1  (light  or  "cold”)  using  MATLAB.  The  munition's 
detonation  event  is  clearly  observable  in  the  image  as  the  dark  object  in  the  upper  right  section  of 
Figure  1. 


A  "change”  image  is  first  created  by  subtracting  the  frame  captured  20  frames 
(~2/3  s)  earlier  from  the  frame  being  analyzed.  As  the  intensities  of  each  of  these  frames  varies 
between  0  and  1,  the  change  image  will  vary  between  -1  (for  dark  or  "hot”  objects)  to  +1  (for 
light  or  "cold”  objects).  The  result  of  performing  this  operation  is  shown  in  Figure  2.  This 
change  detector  generates  an  image  with  a  relatively  neutral  background  (gray)  with  one  large 
dark  (hot)  object  directly  associated  with  the  detonation  event. 

Next,  we  perform  what  we  call  adaptive  thresholding  of  the  change  image  to 
create  a  binary  image.  Adaptive  thresholding  is  required  in  this  application  to  ensure  that 
detonation  events  are  properly  identified  in  the  binary  image.  This  requirement  stems  from  the 
wide  variability  in  intensity  values  as  well  as  other  video  attributes  of  the  munitions'  detonation 
signatures.  The  imagery  shown  later  in  this  section  of  the  report  further  illustrates  this  point. 
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Figure  2.  Change  Image  Generated  from  the  Image  Previously  Shown  in  Figure  1 


We  have  used  the  following  adaptive  threshold  process: 

•  Calculate  the  standard  deviation  of  each  column  in  the  digitized  form  of  the 

change  image. 


•  Calculate  the  median  of  these  standard  deviation  values.  This  value  indicates 
the  “noise  lever  in  the  background  section  of  the  image.  This  is  because  the  detonation 
signature  occurs  over  only  a  few  columns  (compared  to  the  total  number  of  columns)  in  the 
change  image. 
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•  Since  the  detonation  image  always  appears  to  be  darker  (hotter),  we  apply  a 
threshold  to  identify  darker  objects  in  the  change  image.  Specifically,  we  will  look  for 
signatures  that  are  darker  by  nine  standard  deviations  than  the  background  noise  level  in  the 
image.  The  value  nine  was  chosen  since  it  appears  to  consistently  highlight  detonation 
signatures  of  interest  in  the  video  sequences. 

•  We  perform  the  following  operation  to  create  the  binary  image  using  the 
techniques  described  above. 

Binlmage  =  Change  Image  <  -  9  *  [median) change  image  column  standard  deviations)] 

This  operation  sets  all  pixel  values  less  than  the  indicated  threshold  to  I  and  all  pixel  values 
greater  than  or  equal  to  this  threshold  value  to  0. 

The  result  of  performing  the  above  process  on  the  image  previously  shown  in 
Figure  2  is  shown  in  Figure  3.  The  “on  pixels’'’  in  the  binary  image  (Figure  3)  directly 
correspond  to  the  detonation  event  signature.  The  imagery  shown  later  in  this  section  of  the 
report  further  illustrates  the  ability  of  these  methods  to  identify  the  detonation  signature  in  video 
sequences.  Note  that  the  detector  used  in  support  of  the  work  performed  in  References  2  and  3 
uses  the  same  process  (with  one  minor  exception  -  it  also  looks  for  light  objects)  as  described 
above.  This  technique  has  allowed  us  to  reliably  detect  detonation  sequences  in  video  sequences 
regardless  of  the  detonation  signature  intensity  levels  and  the  background  against  which  the 
event  occurs. 


Figure  3.  Binary  Image  Generated  from  the  Change  Image  Previously  Shown  in  Figure  2 


Now,  we  perform  a  morphological  operation  on  the  binary  image.  This  process 
removes  all  objects  <  3  pixels  in  size.  In  the  case  shown  in  Figure  3.  there  are  no  objects  smaller 
than  3  pixels.  Consequently,  this  operation  does  not  alter  this  binary  image. 
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The  final  step  is  to  determine  the  total  number  of  “on”  pixels  in  the  binary  image. 
If  the  total  number  of  “on"  pixels  is  more  than  3,  the  detector  is  considered  “triggered.” 
Otherwise,  the  frame  being  analyzed  is  not  considered  a  frame  of  interest.  The  value  of  3  pixels 
is  selected  as  it  consistently  affords  the  detection  of  signals  of  interest  without  a  significant 
number  of  false  alarms.  This  same  threshold  has  been  used  on  the  DSI  visible  (at  two  focal 
length  settings)  and  1R  camera  data. 

The  classifier  described  later  in  this  report  works  with  up  to  four  frames  (total) 
from  a  video  sequence.  Each  time  the  detector  triggers,  the  frame  number  as  well  as  extracted 
features  are  written  to  a  file.  When  the  first  frame  is  detected,  the  detonation  video  sequence 
starts.  We  examine  10  frames  in  the  detonation  sequence  and  extract  features  from  the  first 
frame  on  which  the  detector  triggers.  We  also  try  to  extract  features  from  the  2nd,  3rd,  and  10th 
frames  in  the  detonation  video  sequence.  If  the  detector  is  not  triggered  on  these  frames,  we 
extract  features  from  the  first  2  frames  on  which  the  detector  triggers  (or  from  less  frames  if  the 
detector  does  not  trigger  two  additional  times  during  the  first  10  frames  of  the  detonation 
sequence).  We  always  try  to  extract  features  from  the  10lh  frame  in  the  detonation  video 
sequence.  When  working  with  this  last  frame,  features  are  extracted  only  if  the  detector  triggers 
on  this  video  frame.  In  general,  for  strong  munition  detonation  signals,  the  detector  will  trigger, 
and  features  will  be  extracted  from  the  1st,  2nd,  3rd,  and  10lh  frames  after  a  munition's  detonation. 
When  the  signatures  are  less  strong,  the  detector  may  only  trigger  on  (and  features  will  be 
extracted  from)  either  the  first  three  frames  after  detonation,  or  the  1 sl,  3rd,  5th.  and  1 0lh  frames  or 
just  on  the  Is1  and  2nd  frames.  There  are  many  possibilities  for  how  the  detector  will  trigger 
during  a  particular  video  sequence,  and  that  the  specifics  of  its  triggering  are  generally  driven  by 
the  specific  characteristics  of  a  munition's  detonation  signature.  Our  experience  has  been  that  in 
the  majority  of  cases  where  the  detector  does  not  trigger  on  all  frames  (or  triggers  on  frames 
other  than  the  1st,  2nd.  3rd.  and  10th  frames)  that  the  detector  still  generally  only  triggers  on  a 
munition's  detonation  signals.  This  is  a  direct  result  of  the  adaptive  thresholding  methods  and 
the  3-pixel  threshold  used  in  this  detector. 

The  following  figures  and  paragraphs  will  provide  images  extracted  from  the 
video  sequences  for  each  of  the  four  classes  of  munitions'  detonation  events.  In  all  of  the 
figures,  the  images  on  the  left  are  extracted  directly  from  the  video  sequence.  The  images  on  the 
right  are  the  binary  images  generated  from  the  images  on  the  left  using  methodologies  described 
above.  In  all  of  the  figures,  the  top  images  are  from  an  HE  airburst.  The  second  row  of  images 
is  from  a  CB  airburst.  The  third  row  of  images  is  from  an  HE  point  detonation,  and  the  final  row 
of  images  is  from  a  CB  point  detonation. 

Figure  4,  5,  6.  and  7  provide  the  images  generated  for  the  four  classes  of 
munitions'  detonation  events  for  the  1st,  2nd,  3rd,  and  10th  frames  after  munitions'  detonations, 
respectively.  These  figures  are  discussed  in  more  detail  in  Section  3,  which  describes  the 
features  used  for  discriminating  between  CB  and  HE  events.  These  features  are  extracted  from 
these  four  video  frames.  We  discuss  how  these  features  are  reduced  to  a  final  feature  vector  for 
each  munition's  detonation  event.  This  feature  vector  becomes  the  input  to  the  classification 
system  described  in  detail  later  in  this  report. 


Figure  4. 


Images  from  the  1  Frame  After  Detonation.  Top  to  Bottom: 
CB  Airburst.  HE  Point  Detonation,  and  CB  Point  Detonation 


HE  Airburst, 
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Figure  5.  Images  from  the  2nd  Frame  After  Detonation.  Top  to  Bottom:  HE  Airburst, 
CB  Airburst.  HE  Point  Detonation,  and  CB  Point  Detonation 
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CB  Airburst,  HE  Point  Detonation,  and  CB  Point  Detonation 
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Figure  7.  Images  from  the  10th  Frame  After  Detonation.  Top  to  Bottom:  HE  Airburst. 
CB  Airburst.  HE  Point  Detonation,  and  CB  Point  Detonation 
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FEATURE  EXTRACTION  AND  FEATURE  VECTOR  DEVELOPMENT 


Each  time  the  detector  "‘triggers"  on  a  video  frame,  a  set  of  features  are  extracted 
from  that  frame.  The  features  from  up  to  four  frames  are  reduced  to  a  single  feature  vector.  This 
vector  is  used  in  a  classifier  described  in  Section  4  of  this  report.  This  section  will  describe  the 
specific  features  extracted  from  each  video  frame  and  will  also  describe  how  the  information 
from  the  video  frames  (up  to  four)  are  condensed  to  create  a  single  feature  vector.  Section  4  will 
also  provide  an  overview  of  how  this  vector  is  used  in  discriminating  between  the  different 
classes  of  munitions'  detonations.  Section  4  will  also  describe  in  detail  the  classifier  that  acts  on 
the  feature  vector  and  will  provide  the  results  of  applying  this  classifier  to  the  AF  IR  video 
imagery  collected  during  the  DSI  field  trial. 

When  the  detector  is  triggered  (by  a  frame  of  interest),  the  binary  image  is 
labeled.  In  cases  where  there  is  only  a  single  object  in  the  binary  image,  features  are  extracted 
from  the  labeled  blob.  The  features  are  extracted  for  each  blob  in  the  image  in  cases  where  there 
are  more  than  one  blob  labeled  in  the  binary  image.  The  following  paragraphs  describe  the 
features  that  are  currently  extracted  from  the  frames  of  interest  (up  to  four)  in  the  AF  IR  video  data. 

A  review  of  the  information  shown  in  Figure  7  clearly  shows  that  detonations  of 
CB  munitions  result  in  objects  of  much  smaller  size  in  the  10th  frame  after  a  munition's 
detonation  compared  to  those  for  HE  munitions.  Consequently,  the  size  of  objects  in  the  binary 
images  provides  a  good  feature  for  discriminating  between  CB  and  HE  munitions.  Figure  7  also 
shows  two  other  points.  First,  there  is  an  orientation  of  blobs  in  the  binary  image  associated  with 
the  HE  air  detonations  that  is  consistent  with  the  azimuth  of  the  incoming  round  and  that  the 
signal-to-noise  ratio  of  the  CB  signatures  is  significantly  smaller  than  that  of  the  HE  signatures. 
This  latter  observation  results  either  in  the  detector  not  triggering  in  sequence  or  in  the  detector 
not  triggering  four  times  when  working  with  CB-generated  video  sequences.  A  review  of  the 
data  shown  in  Figures  5,  6,  and  7  shows  that  the  eccentricity  is  higher  (i.e.,  the  blob  is  more 
elliptical)  in  HE  point  and  air  detonations  when  compared  to  the  CB  point  and  air  detonations. 

Features  are  extracted  from  frames  of  interest  in  each  video  segment.  Features 
extracted  from  the  video  segments  used  for  discrimination  are  the  size  of  objects,  the  orientation 
and  the  eccentricity  of  blobs,  and  whether  or  not  the  detector  triggered  sequentially  when  it  ran 
on  a  particular  video  segment.  Features  based  on  the  grayscale  levels  associated  with  objects 
identified  in  the  binary  image  and  on  the  rate  of  change  of  the  size  of  objects  in  the  binary  image 
between  frames  were  also  evaluated.  However,  these  latter  features  are  not  currently  used  in 
performing  CB  round  discrimination  as  they  did  not  provide  sufficient  new  information  for 
detonation  event  classification. 

The  first  feature  we  extract  from  the  video  frames  of  interest  is  the  size  (number 
of  “on”  pixels)  of  objects  in  the  binary  image.  Primarily,  it  is  this  feature  that  is  used  to 
discriminate  between  CB  and  HE  munition  detonations. 

Next,  we  extract  the  orientation  of  the  blobs  in  the  binary  image.  To  accomplish 
this,  we  render  the  blob  as  an  ellipse  and  compute  the  ellipses  major  and  minor  axes.  The  blob 
orientation  feature  is  the  orientation  of  the  major  axis  with  respect  to  the  horizontal  axis.  In 


16 


cases  where  there  are  multiple  blobs  within  an  image's  frame,  the  minimum  angular  orientation 
of  the  different  blobs  is  used  as  the  orientation  feature  for  the  frame  being  analyzed.  Improved 
classification  performance  is  likely  with  an  improved  means  of  extracting  this  feature  when  there 
are  multiple  blobs  within  an  image  frame. 

Next,  we  extract  an  eccentricity  feature  from  the  blobs  in  the  binary  image.  The 
eccentricity  is  the  ratio  of  the  ellipse  major  axis  and  is  determined  by  first  rendering  the  blob  as 
an  ellipse.  Consequently,  an  object  with  an  eccentricity  of  0  is  circular,  and  an  object  with  an 
eccentricity  of  1  is  a  line.  In  cases  where  there  are  multiple  blobs  present  in  a  frame  of  interest, 
we  average  the  values  of  the  eccentricities  of  all  the  blobs  present  in  the  image  and  use  that  value 
as  the  eccentricity  feature.  Again,  improved  classification  performance  is  likely  with  an 
improved  means  of  extracting  this  feature  in  cases  where  there  are  multiple  blobs  within  an 
image  frame. 

When  the  detector  triggers,  the  test  number  (extracted  from  the  file  name),  frame  number, 
and  each  of  the  features  described  above  are  written  to  a  file.  Table  1  provides  an  example  of 
such  information,  in  this  case  for  an  HE  airburst  event. 


Table  1.  Extracted  Features  from  an  HE  Airburst  Munition  Detonation 


Test  Name 

Frame  Number 

Number  of  “On”  Pixels 

Blob  Orientation 

Blob  Eccentricity 

T258  (#21) 

81 

107 

0 

0.99 

T258  (#21) 

82 

400 

-1.55 

0.88 

T258  (#21) 

83 

638 

-12.84 

0.47 

T258  (#21) 

90 

794 

-42.47 

0.62 

4 

The  number  4  at  the  bottom  of  the  table  indicates  that  the  detector  fired  for  each 
of  the  four  frames  being  analyzed.  As  previously  discussed,  in  some  cases,  the  detector  either 
triggers  on  less  than  four  frames  or  does  not  trigger  on  the  1 st,  2nd,  3rd,  and  1 0,h  frames  after 
detonation.  This  occurs  most  often  for  CB  detonations  and  is  generally  a  result  of  the  lower 
signal-to-noise  ratio  for  this  type  signature  for  this  class  of  munitions'  detonations.  In  all  cases, 
features  such  as  those  presented  in  Table  1  are  combined  to  create  a  single  feature  vector  for  the 
particular  video  segment  being  analyzed.  It  is  this  final  feature  vector  that  is  used  as  input  to  the 
classifiers  described  in  Section  4. 

The  following  steps  describe  how  the  features  are  combined: 

a.  Set  the  sizel  feature  value  to  the  number  of '“on"  (non-zero)  pixels  in  the  last 
frame  on  which  the  detector  triggered.  (Note  that  this  last  frame  can  be  any  frame  within  the 
first  10  frames  after  a  munition's  detonation.  When  the  detector  triggers  four  times,  this 
information  will  always  be  the  10th  frame.  In  other  cases,  the  last  frame  can  be  any  frame  within 
the  first  10  frames.  It  is  the  specifics  of  a  given  video  segment  that  determine  which  frame  this 
will  be). 
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b.  Set  the  Orientation  Feature  to  the  value  associated  with  the  10th  frame  after 
detonation  (fourth  row  of  Table  1).  In  cases  where  the  detector  triggers  less  than  four  times,  the 
Orientation  Feature  is  set  to  zero. 

c.  Set  the  eccentricity  to  the  average  of  the  eccentricity  values  extracted  from  the 
2,Kl  and  3U|  frames  following  a  munition's  detonation.  In  cases  where  there  are  multiple  blobs 
present  in  these  two  frames,  the  eccentricity  feature  value  generated  from  that  frame  is  not 
considered.  In  such  cases,  the  eccentricity  value  is  the  eccentricity  value  extracted  from  the 
frame  where  there  is  only  one  object  present.  In  cases  where  both  of  these  frames  have  more 
than  one  object,  the  eccentricity  value  is  set  to  0. 

d.  Declare  an  event  sequential  when  the  detector  triggers  on  the  1st,  2nd,  3rd,  and 
10th  frames.  Declare  an  event  as  nonsequential  when  this  is  not  the  case. 


Table  2.  Final  Feature  Vector  Generated  from  Table  1  Data 


Test  Name 

Sizel 

Orientation 

Eccentricity 

Sequential 

T258  (#21) 

794 

-42.47 

0.545 

1 

Table  2  shows  an  example  of  a  feature  vector  extracted  from  AF  1R  video 
sequence  T258.  whose  features  are  given  in  Table  1 .  This  feature  vector  is  used  as  input  to  a 
classifier  that  differentiates  between  the  different  classes  of  munitions'  detonations.  The  feature 
vector  extracted  from  visible  camera  video  sequences  and  described  in  detail  in  other  reports2'4 
allowed  discrimination  between  each  of  the  four  classes  of  munitions'  detonations.  For  the  AF 
1R  data,  the  feature  vector  allows  events  to  be  classified  as  HE  airburst,  HE  point  detonation,  CB 
point  detonation,  or  CB  airburst.  The  mode  of  detonation  (either  airburst  or  point  detonation  of 
the  CB  munition),  can  be  determined  independently  by  acoustic  sensors  (triangulation  of  four 
sensors)  as  well  as  from  the  visible  camera  data.  This  further  illustrates  the  importance  of  fusing 
information  from  different  sensors. 

Table  3  provides  the  feature  vectors  extracted  from  each  of  the  available  AF  1R 
video  sequences.  The  class  provided  in  the  right  hand  column  of  the  table  are  either  the  known 
class  (for  the  first  160  rounds)  or  the  estimated  class  based  on  human  review  of  the  AF  IR  and 
DS1  visible  camera"'4  video  sequences.  This  table  also  presents  (at  the  bottom)  a  few  cases 
where  the  class  was  either  unknown  or  where  the  IR  camera  did  not  capture  the  complete 
munition's  detonation  sequence  (e.g.,  the  detonation  signature  was  cropped  by  the  camera's  field 
of  view). 


18 


Table  3.  Feature  Vectors  and  Detonation  Classes  for  AF  IR  Data  Set 


Test  Name 

Sizel 

Orientation 

Eccentricity 

Sequential 

Class 

chemair\T253 

80 

-17  16 

064 

1 

CA 

chemair\T255 

89 

-29  06 

064 

1 

CA 

chemair\T260 

60 

-24  85 

069 

1 

CA 

T173  (#18) 

80 

-52.39 

0.75 

1 

CA 

T175 (#18) 

64 

-53.8 

071 

0 

CA 

T181  (#18) 

64 

-27  82 

0  585 

0 

CA 

T242 (#20) 

106 

-51  73 

0  695 

1 

CA 

T244 (#20) 

167 

-54  87 

083 

1 

CA 

T246 (#20) 

119 

-5205 

0  685 

1 

CA 

T251  (#21) 

91 

-62  21 

0  275 

1 

CA 

T252 (#21) 

62 

-40  02 

0  575 

0 

CA 

chemair\T257 

38 

-7387 

0.55 

0 

CA 

chemair\T259 

39 

13  29 

0  495 

1 

CA 

T164  (#17) 

27 

-1  89 

0  655 

1 

CA 

T209 (#20) 

68 

12  86 

057 

0 

CP 

T215 (#20) 

60 

0 

0565 

1 

CP 

T216 (#20) 

47 

-11  05 

0.52 

0 

CP 

T219 (#21) 

81 

-43  13 

0  165 

1 

CP 

T223  (#21) 

7 

0 

0  76 

1 

CP 

T225 (#21) 

78 

-14.76 

0.53 

1 

CP 

T227 (#22) 

96 

336 

0.52 

1 

CP 

T235  (#22) 

70 

-4  54 

0  585 

1 

CP 

T238  (#22) 

35 

-4  76 

0  9 

1 

CP 

chempointYT  1 14 

291 

10  14 

0.38 

1 

CP 

chempoint\T1 16 

32 

18  43 

0.31 

1 

CP 

chempoint\T127 

152 

0 

0  47 

1 

CP 

T163 (#17) 

52 

-10  29 

058 

1 

CP 

heair/T247 

902 

-30  17 

0575 

1 

HA 

heair\T239 

865 

-49.83 

0505 

1 

HA 

heair\T240 

816 

-34  97 

0  68 

1 

HA 

heair\T241 

815 

-33  34 

0  505 

1 

HA 

heair\T245 

855 

-41  01 

0  525 

1 

HA 

T168 (#17) 

801 

-44  45 

0515 

1 

HA 

T171 (#18) 

837 

-37  07 

0  57 

1 

HA 

T176 (#18) 

820 

-51  45 

0  475 

1 

HA 

T179 (#18) 

756 

-29  14 

052 

1 

HA 

T185 (#18) 

817 

-36  88 

054 

1 

HA 

T249  (#21) 

809 

-4051 

0  545 

1 

HA 

T250  (#21) 

829 

-3277 

0  555 

1 

HA 

T256 (#21) 

855 

-45  11 

0  475 

1 

HA 

T258 (#21) 

794 

-4247 

0  545 

1 

HA 

T222 (#21) 

279 

-0  89 

0.82 

1 

HP 

T174  (#18) 

229 

0  77 

0  73 

1 

HP 

T180  (#18) 

174 

-0  09 

0  86 

1 

HP 

T188 (#18) 

159 

-79 

081 

1 

HP 

T204 (#20) 

310 

3  42 

0785 

1 

HP 

T208  (#20) 

255 

-299 

0  775 

1 

HP 

T210  (#20) 

231 

-2  13 

0  83 

1 

HP 

T21 1 (#20) 

197 

-12.39 

0.8 

1 

HP 

T212 (#20) 

269 

-3  38 

0.85 

1 

HP 

T214  (#20) 

257 

-4  82 

0  795 

1 

HP 

T217  (#21) 

287 

-254 

0815 

1 

HP 

T218  (#21) 

242 

-2.71 

0  805 

1 

HP 

T221 (#21) 

250 

-206 

0  855 

1 

HP 

T224 (#21) 

138 

-593 

0  76 

1 

HP 

T228 (#22) 

132 

-1  81 

09 

1 

HP 

T233 (#22) 

141 

-227 

088 

1 

HP 

T236 (#22) 

249 

-1  86 

09 

1 

HP 

hepoint\T100 

82 

-5  92 

0  72 

1 

HP 

hepoint\T  1 1 2 

188 

-5  44 

0  65 

1 

HP 

hepointYT  1 1 3 

114 

-7.41 

0825 

1 

HP 

hepoint\T1 15 

180 

-0  99 

0.7 

1 

HP 

hepointYT  1 17 

319 

3  25 

0715 

1 

HP 

hepoint\T123 

70 

-1  83 

0745 

1 

HP 

hepointYT  125 

177 

-2  18 

083 

1 

HP 

hepointYT  1 29 

197 

-3.79 

0.76 

1 

HP 

hepoint\T131 

213 

-228 

086 

1 

HP 

T161 

89 

-3  77 

0.89 

1 

HP 

T232 (#22) 

234 

032 

087 

1 

HP 

T205 (#20) 

126 

-4  47 

0.83 

1 

HP 

T177  (#18) 

80 

-24  58 

0  545 

1 

HP  Offscreen 

hepointYT  1  18 

33 

-633 

034 

1 

HP  Offscreen 

T165 (#17) 

143 

-772 

0865 

1 

? 

T234 (#22) 

7 

-1468 

0.85 

0 

? 
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This  section  provides  figures  and  a  discussion  of  how  each  of  the  individual 
features  in  Table  3  are  used  for  discrimination.  The  figures  show  histograms  of  the  feature 
values  for  each  of  the  target  classes  (each  target  class  is  represented  in  a  different  color).  Since 
the  sequential  feature  is  binary  (a  sequence  is  either  sequential  or  not),  a  separate  figure  will  not 
be  generated  for  this  feature.  Instead,  the  data  provided  in  Table  3  will  be  used  to  describe  the 
importance  of  this  feature  in  performing  discrimination. 

As  previously  mentioned,  the  sizel  feature  is  the  most  important  feature  for 
performing  discrimination  when  using  the  AF  IR  video  data.  Figure  8  provides  a  plot  for  the 
histograms  for  each  target  class  for  the  sizel  feature.  The  x  axis  is  the  feature  value,  and  the 
y  axis  is  the  number  of  events  with  that  particular  feature  value.  Each  target  class  is  color  coded. 
Magenta  represents  HE  air  munitions’  detonations.  Blue  represents  HE  point  detonations. 

Green  represents  CB  point  detonations,  and  red  represents  CB  air  detonations.  A  quick  review 
of  the  data  provided  in  Figure  8  clearly  shows  that  the  HE  airbursts  are  fully  separated  from  the 
other  classes  based  solely  on  this  one  feature  value.  The  importance  of  this  feature  in  separating 
the  other  target  classes  is  discussed  in  Figure  9  and  the  next  paragraph. 


Histogram  of  Sizel  Feature  for  4  Classes 
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Figure  8.  Sizel  Feature  Histograms  for  Four  Classes  of  Munitions'  Detonations 
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Histogram  of  Sizel  Feature  for  3  Classes 


Figure  9  shows  the  sizel  feature  histograms  for  HE  point,  CB  point,  and  CB  air 
munitions’  detonations.  Again,  the  x  axis  is  the  sizel  feature  value,  and  the  y  axis  is  the  number 
of  events  with  the  indicated  feature  value.  The  color  coding  is  the  same  as  that  in  Figure  8  (blue 
for  HE  point  detonations,  green  for  CB  point  detonations,  and  red  for  CB  air  detonations.) 

Figure  9  clearly  shows  three  regions  in  the  sizel  feature  space. 

•  Above  a  certain  value  of  sizel  (-160),  all  events  are  HE  point  detonation 
events  (HE  air  detonations  have  the  sizel  values  shown  in  Figure  8). 

•  Below  a  certain  value  of  sizel  (-60),  all  events  are  CB  detonation  events. 

•  Between  these  values  there  is  a  region  with  overlap  between  the  CB 
detonation  events  and  the  HE  point  discharge  events. 

The  classifier  described  in  Section  4  exploits  these  observations.  It  separates  the 
munitions’  detonation  classes  using  just  this  one  feature  where  it  can  and  uses  other  features 
when  the  sizel  feature  value  is  in  the  overlap  region. 
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Histograms  for  the  orientation  feature  are  provided  in  Figure  10.  Because  the  HE 
airbursts  can  be  fully  separated  from  the  other  classes  just  by  using  the  sizel  feature,  the 
Orientation  Feature  histograms  are  provided  for  the  HE  point,  CB  point,  and  CB  air  detonations 
only.  In  Figure  10,  blue  represents  HE  point  detonations,  green  represents  CB  point  detonations, 
and  red  represents  CB  air  detonations.  The  data  in  Figure  10  clearly  show  the  ability  to  provide 
some  separation  of  the  CB  airbursts  (red)  from  the  CB  point  detonations  (green)  and  the  HE 
point  detonations  (blue)  using  this  feature.  This  observation  is  also  used  in  the  classifier 
described  in  Section  4. 
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Histogram  of  Orientation  Feature  for  3  Classes 
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Figure  10.  Orientation  Feature  Histograms  for  Three  Classes  of  Munitions’  Detonations 


Figure  1 1  shows  histograms  for  the  eccentricity  feature  for  the  same  three  classes. 
Again,  blue  represents  HE  point  detonations,  green  represents  CB  point  detonations,  and  red 
represents  CB  airbursts.  Figure  1 1  also  shows  the  ability  to  separate  the  HE  point  detonation 
class  from  either  the  CB  point  or  CB  airburst  classes  using  this  feature.  The  classifier  described 
in  Section  4  also  exploits  this  observation. 
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Histogram  of  Eccentricity  Feature  for  3  Classes 


Figure  1 1.  Eccentricity  Feature  Histograms  for  Three  Classes  of  Munitions’  Detonations 


Histograms  for  the  sequential  feature  are  not  generated  as  this  is  a  binary  feature. 
However,  the  information  provided  in  Table  3  can  be  used  to  determine  the  value  of  this  feature 
in  performing  discrimination.  In  every  case  evaluated,  the  triggering  was  sequential  for  the  HE 
air  and  HE  point  munitions’  detonations.  In  6  cases  out  of  27,  CB  munitions’  detonations  were 
not  sequential.  The  classifier  described  in  Section  4  also  exploits  this  observation. 


4.  CLASSIFIER  DESCRIPTIONS  AND  RESULTS 

The  Linguistic-Fuzzy  Classifier  approach  previously  described  in  detail  '  was  used 
to  evaluate  the  feature  vector  and  to  generate  confidence  values  for  discrimination.  In  this  case, 
we  have  developed  a  hierarchical  classifier  that  first  tries  to  use  the  sizel  feature  alone  for 
performing  classification. 
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The  following  descriptions  are  used  for  the  target  classes: 

•  HE  air  detonations  have  very  large  sizel  values 

•  HE  point  detonations  have  large  sizel  values 

•  CB  detonations  have  small  sizel  values 

•  Detonations  where  we  need  more  information  occur  over  an  in-between 
range  of  sizel  values. 

Figure  12  shows  sizel  feature  values  and  membership  functions  for  all  four 
classes  of  munitions'  detonations.  Figure  13  shows  the  sizel  feature  histograms  and  membership 
functions  for  the  region  where  there  is  more  class  overlap.  In  both  figures,  the  blue  histograms 
represent  the  HE  munitions  (both  point  and  air),  and  the  red  histograms  represent  the  CB 
munitions.  The  green  membership  function  defines  very  large  sizel  feature  values  in  the  class 
descriptions  above  (only  shown  in  Figure  12).  The  yellow  membership  function  defines  large 
sizel  feature  values,  and  the  magenta  membership  function  defines  small  sizel  feature  values  in 
the  class  descriptions  above.  Lastly,  the  cyan  membership  function  defines  the  in-between 
range  of  sizel  in  the  class  descriptions  above. 


1000 

Figure  12.  Histograms  and  Membership  Functions  for  the  Sizel  Feature 
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Sizel  for  3  classes  with  Initial  Membership  Functions 
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Figure  13.  Histograms  and  Membership  Functions  for  the  Sizel  Feature  (Expanded  Scale) 


To  generate  confidence  values  using  the  sizel  feature  value,  we  first  determine  to 
which  membership  function  the  particular  value  of  the  sizel  feature  belongs  and  use  this  to 
generate  the  confidence  value.  For  example,  if  the  sizel  feature  value  is  800,  than  the 
intersection  with  the  very  large  sizel  membership  function  is  1,  and  the  intersection  with  all 
other  membership  functions  is  zero  (Figure  12).  This  results  in  a  100%  confidence  value  that 
this  is  an  HE  air  detonation.  Similarly,  a  sizel  value  of  30  results  in  an  intersection  of  1  with 
the  small  sizel  membership  function  and  an  intersection  of  zero  with  all  other  membership 
functions. 


Consequently,  this  results  in  a  100%  confidence  value  that  this  is  a  CB 
detonation.  Finally,  if  the  sizel  feature  value  is  160,  we  have  an  intersection  of  approximately 
0.3  with  the  large  sizel  membership  function  and  an  intersection  of  1  with  the  in-between  range 
of  sizel  membership  function.  In  this  case,  we  would  classify  the  event  as  a  detonation  where 
we  need  more  information,  and  other  features  would  be  evaluated  by  different  linguistic-fuzzy 
classifiers  to  generate  a  confidence  value  that  would  be  used  for  discrimination.  In  cases  where 
the  sizel  feature  value  results  in  confidence  values  being  generated  for  CB  detonation, 
detonation  where  we  need  more  information,  or  HE  point  detonation,  the  higher  confidence 
value  is  used  to  determine  the  classification  path.  In  cases  where  either  the  CB  detonation  or 
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HE  point  detonation  confidences  are  either  greater  than  or  equal  to  the  detonation  where  we 
need  more  information  confidence  value,  either  the  CB  or  HE  point  detonation  confidence 
value  is  used  as  the  final  output  for  the  classifier.  In  cases  where  the  detonation  where  we  need 
more  information  confidence  value  is  greater  than  either  the  CB  or  HE  point  detonation 
confidences,  the  next  layer  of  the  classifier  (described  below)  is  used  to  generate  the  final  output 
for  the  classifier. 

For  these  interim  cases,  we  use  the  following  descriptions  of  the  target  classes: 

•  HE  point  detonations  have  higher  size I  values,  higher  eccentricity  values, 
and  near-zero  orientations. 

•  CB  detonations  have  lower  sizel  values,  lower  eccentricity  values,  more 
negative  orientation  values,  and  are  not  always  sequential. 

Figure  14  shows  the  histograms  and  membership  functions  used  in  this  layer  of 
the  classifier  (excluding  the  sequential  feature  -  its  importance  was  described  in  Section  3.) 


Figure  14.  Histograms  and  Membership  Functions  Used  in  the  Second  Classifier  Layer 
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In  Figure  14,  the  plot  on  the  upper  left  is  for  the  sizel  feature.  The  plot  on  the 
upper  right  is  for  the  eccentricity  feature,  and  the  plot  in  the  lower  center  is  for  the  orientation 
feature.  In  these  plots,  the  blue  histograms  represent  HE  point  detonations,  and  the  red 
histograms  represent  CB  point  and  air  detonations.  The  membership  functions  shown  for  the 
sizel  feature  cover  the  same  range  as  the  in-between  range  of  sizel  membership  function 
described  earlier  in  Figure  13.  The  magenta  membership  function  defines  lower  sizel  in  the 
class  descriptions  provided  above,  while  the  cyan  membership  function  defines  higher  sizel  in 
the  class  descriptions.  The  magenta  membership  function  in  the  eccentricity  feature  space 
defines  lower  eccentricity,  while  the  cyan  membership  function  defines  higher  eccentricity.  The 
magenta  membership  function  in  the  orientation  space  defines  more  negative  orientation,  while 
the  cyan  membership  function  defines  near-zero  orientation. 

It  is  important  to  keep  in  mind  that  the  second  layer  of  the  classifier  is  only  used 
in  cases  where  a  class  declaration  cannot  be  made  based  on  the  first  layer  of  the  classifiers’ 
assessment  of  sizel  feature  value  (i.e.,  when  sizel  is  neither  too  high  nor  too  low,  but  falls  in  the 
mid  range).  This  second  layer  classifier  is  used  in  <  30%  of  the  cases  that  were  evaluated.  A 
review  of  the  histograms  and  membership  functions  provided  in  Figure  14  shows  that  the  classes 
cannot  be  easily  separated  using  the  single  features  and  membership  functions  shown  in  the 
figure.  For  example,  the  CB  and  HE  point  detonations  have  orientations  that  are  in  the  same 
range  of  angles  (near  zero).  This  becomes  apparent  when  the  data  previously  presented  in 
Figure  10  is  considered  with  that  provided  in  Figure  14.  Consequently,  this  feature  is  more 
valuable  in  separating  the  CB  air  detonations  from  the  HE  point  detonations  (as  well  as  from  the 
CB  point  detonations).  As  a  second  example,  a  review  of  the  eccentricity  feature  values  and 
membership  functions  shows  that  this  feature  is  more  useful  in  separating  the  CB  point 
detonations  from  the  HE  point  detonations.  But,  the  feature  is  less  useful  in  separating  CB  air 
detonations  from  HE  point  detonations.  This  becomes  apparent  when  the  data  previously 
presented  in  Figure  1 1  is  considered  with  that  provided  in  Figure  14.  There  is  better  separation 
between  the  CB  (point  and  air)  detonations  and  the  HE  point  detonations  with  the  sizel  feature 
data  shown  in  Figure  14.  However,  there  is  still  some  overlap  between  the  classes  (considering 
the  membership  functions  shown  in  Figure  14)  in  this  feature  space.  As  previously  mentioned  in 
this  report,  6  of  the  27  CB  detonation  (and  0  of  44  HE  detonation)  sequences  evaluated  produced 
nonsequential  triggering  of  the  detector.  Again,  this  feature  can  contribute  towards  confidence 
estimation,  but  it  should  not  independently  drive  the  final  class  declaration. 

Since  there  is  only  fair  separation  between  the  classes  using  single  features  in  this 
second  layer  classifier,  it  is  a  good  opportunity  to  demonstrate  the  value  of  data  (or  feature) 
fusion  to  achieve  better  discrimination  between  the  classes.  We  use  the  Linguistic-Fuzzy 
Classifier4  for  that  purpose.  We  configured  the  linguistic-fuzzy  classifier  in  a  manner  that 
requires  at  least  two  of  the  three  major  attributes  to  have  a  strong  membership  in  the  appropriate 
membership  functions  to  drive  the  confidence  value  to  a  level  higher  than  0.5.  This  is 
accomplished  through  the  proper  selection  of  weights  for  each  of  the  attributes  used  in  the 
classifier.  The  sequential  feature  is  a  less  important  attribute  for  discrimination.  Therefore,  for 
this  feature,  a  weight  was  selected  that  results  in  only  a  small  increase  (approximately  0.08)  in 
confidence  value  for  CB  detonations  when  the  video  sequence  results  in  the  detector  triggering  in 
a  nonsequential  manner. 
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Using  these  considerations  as  guide,  the  following  relationship  is  used  to  estimate 
a  confidence  value  for  HE  point  detonations: 

CONFuEpoin,  =  (W higher sizei*Mem(higher  sizel))2  +  (W higher  eccen,ricity*Mcm(higher  eccentricity ))  ( 1 ) 

F  (^orientation  near  zt’ro*  ^\tZY\\{Of'ictl{Utiotl  tlCll)  zero ) )  AW  higher  sizel~  higher  eccentricity 

^^orientation  near  zero  ) 


where 

=  confidence  that  the  observed  detonation  is  an  HE  point  munition’s 
detonation  -  maximum  value  is  1  (100%  confidence)  and  minimum  value 
is  0  (0%  confidence) 

=  weight  for  the  sizel  feature  for  performing  HE  detonation 
discrimination  =  0.5 

=  weight  for  the  eccentricity  feature  for  performing  HE  detonation 
discrimination  =  0.5 

=  weight  for  the  orientation  feature  for  performing  HE  detonation 
discrimination  =  0.5 

=  membership  of  the  sizel  feature  value  in  the  higher  sizel  membership 
function  shown  in  Figure  14 

=  membership  of  the  eccentricity  feature  value  in  the  higher  eccentricity 
membership  function  shown  in  Figure  14 

=  membership  of  the  orientation  feature  value  in  the  orientation  near  zero 
membership  function  shown  in  Figure  14 

The  membership  functions  used  in  this  classifier  were  selected  based  on  analyses 
of  the  histograms  provided  in  Figure  14.  The  weights  for  the  individual  features  were  selected 
based  on  the  considerations  discussed  earlier  in  this  report. 

The  following  relationship  is  used  to  estimate  a  confidence  value  for  CB 

detonations: 

CONFcb  =  (W lover sizei * Mcm( lower  Sizel))2  +  ( W/oH*r ^,W(F*Mem(to^r  eccentricity))  “  (2) 

+  (W  nwre  negative  orientation*Mem(mOre  negative  orientation ))~  +  (  W  not  sequential  *  Mem 

(not  sequential))  /  (w  lover  Sizel  +  W lover  eccentricity  +W  more  negative  orientation  ^  not  sequential  ) 

where 

CONFcb  =  confidence  that  the  observed  detonation  is  a  CB  munition’s  detonation 

(point  or  airburst)  -  maximum  value  is  1  (100%  confidence)  and 
minimum  value  is  0  (0%  confidence) 


CONFiit  point 


WA/1 


higher  size 1 


W  higher  eccentricity' 


w„„, 


orientation  near  zero 


Mem  (higher  sizel) 


Mem  (higher  eccentricity) 


Mem  (orientation  near  zero) 
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w 


lon  er  size  I 


=  weight  for  the  sizel  feature  for  performing  CB  detonation 
discrimination  =  0.5 


W 


lower  eccentricity 


=  weight  for  the  eccentricity  feature  for  performing  CB  detonation 
discrimination  =  0.5 


W 


not  sequential 


=  weight  for  the  eccentricity  feature  for  performing  CB  detonation 
discrimination  =  0.25 


Mem(/owr  sizel) 


=  membership  of  the  sizel  feature  value  in  the  lower  sizel  membership 
function  shown  in  Figure  14 


Mem(/m»'t’r  eccentricity)  =  membership  of  the  eccentricity  feature  value  in  the  higher  eccentricity' 

membership  function  shown  in  Figure  14 


Mem(more  negative  =  membership  of  the  orientation  feature  value  in  the  more  negative 

orientation)  orientation  membership  function  shown  in  Figure  14 


Mcm(n»t  sequential)  =  I  if  the  detector  triggered  not  sequentially  on  the  video  sequence 

(i.c.,  the  sequential  feature  is  =  0)  or  =  0  if  the  detector  triggered 
sequentially 


As  in  the  case  described  earlier,  the  membership  functions  used  in  this  classifier 
resulted  from  analyses  performed  on  the  histograms  provided  in  Figure  14.  The  weights  were 
selected  for  the  reasons  previously  discussed  in  detail  in  this  section  of  the  report. 


section. 


Section  5  will  provide  the  results  obtained  using  the  classifiers  described  in  this 


5.  LINGUISTIC-FUZZY  CLASSIFIER  RESULTS  SUMMARY 

Table  4  gives  the  results  of  applying  the  detector,  feature  extraction,  and 
linguistic-fuzzy  classifiers  to  each  of  the  available  AF  IR  DSI  video  sequences.  The  table  also 
provides  the  file  name,  the  feature  vector  (sizel,  orientation,  eccentricity,  and  sequential  status), 
the  linguistic-fuzzy  classifier  confidence  values  for  HE  air  detonation,  HE  point  detonation  and 
CB  detonation,  the  final  class  declaration,  and  a  comment  field.  The  final  class  declarations  are 
based  on  the  confidence  values  provided  in  this  table.  The  highest  confidence  value  of  the  three 
outputs  from  the  linguistic-fuzzy  classifier  determines  the  final  class  declaration.  When  the  two 
highest  confidence  values  were  within  0.1  of  each  other,  that  event  would  be  declared 
indeterminate  based  on  the  classifier  outputs  (note  this  did  not  happen  with  any  of  the  feature 
vectors  extracted  from  the  74  AF  IR  video  sequences). 
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Table  4.  Linguistic-Fuzzy  Classifier  AF  IR  Results  Summary 


Test  Name 

Sizel 

Orientation 

Eccentricity 

Sequential 

HE  Air  Conf 

HE  Point  Conf 

CB  Conf 

Class 

Comments 

chemair\T253 

80 

-17  16 

064 

1 

0 

0 

1 

CB 

chemair\T255 

89 

-29  06 

064 

1 

0 

0 

0.96 

CB 

chemair\T260 

60 

-24  85 

0  69 

1 

0 

0 

1 

CB 

T173 (#18) 

80 

-52  39 

0  75 

1 

0 

0 

1 

CB 

T175  (#18) 

64 

-53  8 

071 

0 

0 

0 

1 

CB 

T181 (#18) 

64 

-27  82 

0  585 

0 

0 

0 

1 

CB 

T242  (#20) 

106 

-51  73 

0  695 

1 

0 

0 

0.96 

CB 

T244  (#20) 

167 

-54  87 

0  83 

1 

0 

0.82 

0  55 

HP  (CB) 

Event  Obscured  by  Reticule 

T246  (#20) 

119 

-52  05 

0  685 

1 

0 

0 

0.96 

CB 

T251  (#21) 

91 

-6221 

0  275 

1 

0 

0 

0.96 

CB 

T252  (#21) 

62 

40  02 

0  575 

0 

0 

0 

1 

CB 

chemair\T257 

38 

-73  87 

0  55 

0 

0 

0 

1 

CB 

chemairtT259 

39 

13.29 

0495 

1 

0 

0 

1 

CB 

T164  (#17) 

27 

-1  89 

0655 

1 

0 

0 

1 

CB 

T209 (#20) 

68 

12  86 

0  57 

0 

0 

0 

1 

CB 

T215  (#20) 

60 

0 

0  565 

1 

0 

0 

1 

CB 

T216  (#20) 

47 

-11  05 

0  52 

0 

0 

0  58 

083 

CB 

T219  (#21) 

81 

43  13 

0  165 

1 

0 

0 

0.9 

CB 

T223  (#21) 

7 

0 

0  76 

1 

0 

0 

1 

CB 

T225 (#21) 

78 

-14  76 

053 

1 

0 

0 

1 

CB 

T227 (#22) 

96 

3  36 

052 

1 

0 

0  58 

0.78 

CB 

T235  (#22) 

70 

454 

0  585 

1 

0 

0 

1 

CB 

T238  (#22) 

35 

4  76 

09 

1 

0 

0 

1 

CB 

chempointVT1 14 

291 

10  14 

0  38 

1 

0 

1 

0 

HP  (CB) 

False  Negative 

chempointvn  16 

32 

1843 

031 

1 

0 

0 

1 

CB 

chempoint\T127 

152 

0 

047 

1 

0 

0.82 

055 

HP(CB) 

False  Negative 

T163 (#17) 

52 

-1029 

0  58 

1 

0 

0 

1 

CB 

heair/T247 

902 

-3017 

0  575 

1 

1 

0 

0 

HA 

heair\T239 

865 

49  83 

0  505 

1 

1 

0 

0 

HA 

heai  AT 240 

816 

-34  97 

0  68 

1 

1 

0 

0 

HA 

heair\T241 

815 

-33  34 

0  505 

1 

1 

0 

0 

HA 

heair\T245 

855 

41  01 

0  525 

1 

1 

0 

0 

HA 

T168 (#17) 

801 

44  45 

0  515 

1 

1 

0 

0 

HA 

T171 (#18) 

837 

-37  07 

0  57 

1 

1 

0 

0 

HA 

T176 (#18) 

820 

-51  45 

0475 

1 

1 

0 

0 

HA 

T179  (#18) 

756 

-29  14 

052 

1 

1 

0 

0 

HA 

T185 (#18) 

817 

-36  88 

054 

1 

1 

0 

0 

HA 

T249(#21) 

809 

4051 

0  545 

1 

1 

0 

0 

HA 

T250 (#21) 

829 

-32  77 

0  555 

1 

1 

0 

0 

HA 

T256 (#21) 

855 

4511 

0475 

1 

1 

0 

0 

HA 

T258  (#21) 

794 

4247 

0  545 

1 

1 

0 

0 

HA 

T222  (#21) 

279 

-0  89 

0  82 

1 

0 

1 

0 

HP 

T174 (#18) 

229 

0.77 

0  73 

1 

0 

1 

0 

HP 

T180 (#18) 

174 

-0  09 

0  86 

1 

0 

1 

0 

HP 

T188  (#18) 

159 

-7  9 

081 

1 

0 

1 

0 

HP 

T204  (#20) 

310 

3  42 

0785 

1 

0 

1 

0 

HP 

T208  (#20) 

255 

-299 

0775 

1 

0 

1 

0 

HP 

T210  (#20) 

231 

-2  13 

083 

1 

0 

1 

0 

HP 

T21 1 (#20) 

197 

-1239 

0.8 

1 

0 

1 

0 

HP 

T212 (#20) 

269 

-3  38 

0  85 

1 

0 

1 

0 

HP 

T214 (#20) 

257 

482 

0795 

1 

0 

1 

0 

HP 

T217 (#21) 

287 

-254 

0815 

1 

0 

1 

0 

HP 

T218 (#21) 

242 

-271 

0805 

1 

0 

1 

0 

HP 

T221 (#21) 

250 

-2  06 

0  855 

1 

0 

1 

0 

HP 

T224 (#21) 

138 

-5  93 

0  76 

1 

0 

1 

0 

HP 

T228 (#22) 

132 

-1  81 

09 

1 

0 

1 

0 

HP 

T233 (#22) 

141 

-227 

0  88 

1 

0 

1 

0 

HP 

T236  (#22) 

249 

-186 

09 

1 

0 

1 

0 

HP 

hepoinlVTIOO 

82 

-5  92 

072 

1 

0 

0 

0.8 

CB  (HP) 

False  Positive 

hepointM  112 

188 

-544 

065 

1 

0 

1 

0 

HP 

hepoi  nlM  113 

114 

-741 

0825 

1 

0 

0.82 

0  55 

HP 

hepointM  115 

180 

-0  99 

07 

1 

0 

1 

0 

HP 

hepointM  1 1 7 

319 

3.25 

0715 

1 

0 

1 

0 

HP 

hepoi  nt\T1 23 

70 

-183 

0745 

1 

0 

0 

1 

CB  (HP) 

False  Positive 

hepoi  nt\T1 25 

177 

-218 

083 

1 

0 

1 

0 

HP 

hepointM  129 

197 

-3.79 

076 

1 

0 

1 

0 

HP 

hepoi  ntMl  31 

213 

-228 

086 

1 

0 

1 

0 

HP 

T161 

89 

-3  77 

0  89 

1 

0 

0.82 

055 

HP 

T232 (#22) 

234 

0  32 

087 

1 

0 

1 

0 

HP 

T205  (#20) 

126 

4  47 

0  83 

1 

0 

1 

0 

HP 

T177 (#18) 

80 

-24  58 

0  545 

1 

0 

0 

1 

CB  (HP) 

HP  Edge  of  FOV 

hepointVT  118 

33 

-63  3 

034 

1 

0 

0 

1 

CB  (HP) 

HP  Edge  of  FOV 

T165 (#17) 

143 

-772 

0865 

1 

0 

1 

0 

HP 

T234 (#22) 

7 

-14  68 

0  85 

0 

0 

0 

1 

CB 
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Overall,  excellent  discrimination  results  have  been  obtained  using  the  feature 
vectors  and  linguistic-fuzzy  classifiers  described  herein.  Considering  the  results  from  all  74 
video  sequences,  the  linguistic-fuzzy  classifiers  correctly  classified  the  detonation  events  91%  of 
the  time.  Three  CB  detonation  events  were  declared  HE  point  detonations  (false  negatives),  and 
four  HE  point  detonations  were  declared  CB  detonations  (false  positives). 

Please  note  that  in  two  HE  point  detonation  events,  the  detonation  signatures  were 
at  the  very  edge  of  the  field  of  view  of  the  camera  and  that  the  feature  values  for  sizel  and 
eccentricity  were  corrupted  since  the  signature  in  the  image  was  cropped.  Specifically,  this 
caused  these  two  events  to  be  declared  CB  detonations.  The  two  events  are  noted  in  Table  4.  In 
addition,  one  of  the  CB  airburst  detonation's  signature  was  corrupted  by  video  overlay  reticule. 
This  corrupted  the  eccentricity  feature  value  and  drove  the  declaration  of  this  video  sequence  to 
HE  point  detonation.  If  the  three  events  were  eliminated,  the  linguistic  fuzzy  classifier  correctly 
classified  detonations  in  94%  of  the  cases  with  two  false  positives  and  two  false  negatives. 

These  results  clearly  demonstrate  the  feasibility  of  using  1R  imaging  systems  to  detect  CB 
munitions'  use.  In  addition,  this  discrimination  can  occur  well  within  1  s  of  a  munition's 
detonation  (including  feature  extraction,  feature  evaluation,  and  final  event  type  declaration). 

A  secondary  benefit  of  the  linguistic-fuzzy  classifiers  described  in  this  report  is 
that  the  confidence  levels  (as  well  as  either  some  or  all  of  the  features  used  to  generate  the 
confidence  values)  can  be  used  as  input  to  a  multisensor  data  fusion  engine.  For  example,  the 
confidence  values  and  either  some  or  all  of  the  feature  values  provided  in  Table  4  could  be 
combined  with  the  confidence  values  and  either  some  or  all  of  the  feature  values  extracted  from  a 
visible  camera  video  sequences.4  This  will  be  the  topic  of  a  future  report.  Also,  confidence  and 
feature  values  from  the  imaging  systems  (IR  and  visible)  can  also  be  combined  with  those 
obtained  using  acoustic  and  seismic  sensors  that  are  also  part  of  the  DSI  sensor  suite.  This  also 
will  be  the  topic  of  a  future  report  because  it  is  likely  that  it  will  increase  the  accuracy  of  the  CB 
detection/early  warning  process. 


6.  CONCLUSIONS 

In  this  report,  we  describe  the  methods  used  to  identify  frames  of  interest  and  to 
extract  features  from  the  Disparate  Sensor  Integration  (DSI)  Air  Force  (AF)  IR  Video  Sequences. 
We  also  describe  how  the  features  extracted  from  individual  frames  are  combined  to  create  a 
single  feature  vector.  These  processes  are  completed  in  the  first  10  video  frames  (0.33  s)  after  a 
munition's  detonation  event  is  first  detected.  We  also  describe  a  unique  classifier  system  that  we 
used  to  evaluate  the  feature  vectors  and  to  develop  confidence  values  for  HE  air  detonations,  HE 
point  detonations,  and  CB  detonations.  We  show  how  we  can  use  these  confidence  values  to 
detect  CB  munitions'  use. 

When  we  apply  these  methods  to  the  DSI  AF  IR  video  data  set,  we  achieved  a 
very  high  success  rate  in  identifying  CB  munitions.  If  we  consider  video  sequences  where  the 
detonation  signatures  are  not  corrupted  by  video  artifacts  (cropping),  we  were  able  to  correctly 
identify  the  munition's  fill  (either  CB  or  HE)  in  94%  of  the  cases  evaluated.  In  the  71  video 
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sequences  evaluated,  there  were  two  false  positives  and  two  false  negatives.  These  results 
clearly  demonstrate  the  potential  for  detecting  CB  rounds  using  1R  imaging  systems.  In  addition, 
the  methods  used  will  allow  discrimination  to  occur  well  within  1  s  of  a  munition's  detonation. 

Future  work  will  be  directed  at  fusing  the  DSI  AF  IR  imaging  system  results  with 
visible  imaging  system  results.  The  DSI  Dugway  Proving  Ground  (DPG)  visible  imaging  system 
confidence  outputs  and  features'  vectors  are  described  in  detail.4  This  work  will  use  the 
confidence  values  and  either  some  or  all  of  the  feature  values  (described  herein  and  in 
Reference  4)  as  inputs  to  a  multisensor  data  fusion  engine.  We  anticipate  that  further 
improvements  in  classification  results  will  be  obtained  when  this  next  level  of  data  fusion  is 
performed. 


Future  work  will  also  be  directed  at  fusing  the  results  obtained  with  the  visible 
and  IR  imaging  systems  with  those  obtained  using  the  DSI  acoustic  and  seismic  sensors.  As 
these  sensors  provide  a  more  orthogonal  detection  modality,  we  anticipate  that  performing 
multisensor  data  fusion  at  this  level  will  have  significant  pay  offs  in  classification  results  and 
robustness  of  final  target  declarations.  We  have  also  initiated  a  modeling  and  simulation  effort 
that  will  be  used  to  predict  performance  of  a  DSI  sensor  suite  in  more  complex  simulated 
measurement  environments.  The  simulated  environments  include  scenarios  where  the  detonation 
events  occur  at  multiple  locations  with  respect  to  the  sensor  locations  as  well  as  environments 
where  multiple  detonation  events  occur  nearly  simultaneously.  The  modeling  efforts  will  also  be 
used  to  help  define  the  robustness  of  individual  classifiers  at  the  sensor  levels  as  well  as  fusing 
different  sensor  types.  Through  simulation,  we  will  be  able  to  demonstrate  the  many  benefits 
that  are  likely  to  be  obtained  through  performing  multisensor  data  fusion. 

The  methods  described  herein  provide  a  computationally  efficient  and  robust 
means  for  detection  and  early  warning  when  CB  munitions  are  used. 
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